Skip to content

Converts prepoceanobs to use JCB-style obs list and config files#1527

Merged
AndrewEichmann-NOAA merged 10 commits into
developfrom
feature/jcb-obs-list
Mar 11, 2025
Merged

Converts prepoceanobs to use JCB-style obs list and config files#1527
AndrewEichmann-NOAA merged 10 commits into
developfrom
feature/jcb-obs-list

Conversation

@AndrewEichmann-NOAA
Copy link
Copy Markdown
Collaborator

@AndrewEichmann-NOAA AndrewEichmann-NOAA commented Mar 6, 2025

Description

This converts the prepoceanobs task to using an obs list that ingests the jinja yaml observation configs in jcb-gdas. Also attempts to align script-internal variable names with existing JEDI yaml nomenclature. Similar changes will also be incorporated into NOAA-EMC/global-workflow#3381

Companion PRs

None

Issues

Resolves #1285

Automated CI tests to run in Global Workflow

  • C48mx500_3DVarAOWCDA
  • C48mx500_hybAOWCDA

@AndrewEichmann-NOAA
Copy link
Copy Markdown
Collaborator Author

Ok, tested with new argo configs

@AndrewEichmann-NOAA AndrewEichmann-NOAA added the hera-GW-RT Queue for automated testing with global-workflow on Hera label Mar 7, 2025
@emcbot emcbot added hera-GW-RT-Running Automated testing with global-workflow running on Hera and removed hera-GW-RT Queue for automated testing with global-workflow on Hera labels Mar 7, 2025
Comment thread parm/soca/obs/obs_list.yaml.j2
Comment thread parm/soca/obs/obs_list.yaml.j2
@emcbot
Copy link
Copy Markdown

emcbot commented Mar 7, 2025

Automated GW-GDASApp Testing Results:
Machine: hera

Start: Fri Mar  7 14:42:26 UTC 2025 on hfe03
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Fri Mar  7 15:27:28 UTC 2025
---------------------------------------------------
Tests: ctest -j12 -R gdasapp -E atm_jjob|C96C48_ufs_hybatmDA|C96C48_hybatmaerosnowDA|C96C48_hybatmDA
Tests:                                  *Failed*
Tests: Failed at Fri Mar  7 21:58:32 UTC 2025
Tests: 80% tests passed, 11 tests failed out of 56
	2063 - test_gdasapp_C48mx500_3DVarAOWCDA_gdas_fcst_202103241800 (Failed)
	2066 - test_gdasapp_C48mx500_3DVarAOWCDA_gdas_marineanlinit_202103250000 (Failed)
	2067 - test_gdasapp_C48mx500_3DVarAOWCDA_gdas_marineanlvar_202103250000 (Failed)
	2068 - test_gdasapp_C48mx500_3DVarAOWCDA_gdas_marineanlchkpt_202103250000 (Failed)
	2069 - test_gdasapp_C48mx500_3DVarAOWCDA_gdas_marineanlfinal_202103250000 (Failed)
	2078 - test_gdasapp_C48mx500_hybAOWCDA_gdas_marineanlinit_202103250000 (Failed)
	2079 - test_gdasapp_C48mx500_hybAOWCDA_gdas_marineanlvar_202103250000 (Failed)
	2081 - test_gdasapp_C48mx500_hybAOWCDA_gdas_marineanlchkpt_202103250000 (Failed)
	2082 - test_gdasapp_C48mx500_hybAOWCDA_gdas_marineanlfinal_202103250000 (Failed)
	2065 - test_gdasapp_C48mx500_3DVarAOWCDA_gdas_marinebmat_202103250000 (Timeout)
	2076 - test_gdasapp_C48mx500_hybAOWCDA_gdas_marinebmat_202103250000 (Timeout)
Tests: see output at /scratch1/NCEPDEV/da/role.jedipara/CI/GDASApp/workflow/PR/1527/global-workflow/sorc/gdas.cd/build/log.ctest

@emcbot emcbot added hera-GW-RT-Failed Automated testing with global-workflow failed on Hera and removed hera-GW-RT-Running Automated testing with global-workflow running on Hera labels Mar 7, 2025
Copy link
Copy Markdown
Collaborator

@apchoiCMD apchoiCMD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Look good- and ci test again?

@AndrewEichmann-NOAA
Copy link
Copy Markdown
Collaborator Author

Look good- and ci test again?

The results of the previous failure are strange - the fcst log returns 0, and marineanlinit has the gridspec.nc problem

@AndrewEichmann-NOAA AndrewEichmann-NOAA added hera-GW-RT Queue for automated testing with global-workflow on Hera and removed hera-GW-RT-Failed Automated testing with global-workflow failed on Hera labels Mar 11, 2025
@emcbot emcbot added hera-GW-RT-Running Automated testing with global-workflow running on Hera and removed hera-GW-RT Queue for automated testing with global-workflow on Hera labels Mar 11, 2025
@emcbot
Copy link
Copy Markdown

emcbot commented Mar 11, 2025

Automated GW-GDASApp Testing Results:
Machine: hera

Start: Tue Mar 11 10:58:51 UTC 2025 on hfe08
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Tue Mar 11 11:43:13 UTC 2025
---------------------------------------------------
Tests: ctest -j12 -R gdasapp -E atm_jjob|C96C48_ufs_hybatmDA|C96C48_hybatmaerosnowDA|C96C48_hybatmDA
Tests:                                 *SUCCESS*
Tests: Completed at Tue Mar 11 12:04:45 UTC 2025
Tests: 100% tests passed, 0 tests failed out of 56

@emcbot emcbot added hera-GW-RT-Passed Automated testing with global-workflow successful on Hera and removed hera-GW-RT-Running Automated testing with global-workflow running on Hera labels Mar 11, 2025
@AndrewEichmann-NOAA
Copy link
Copy Markdown
Collaborator Author

@guillaumevernieres Passed!

@AndrewEichmann-NOAA AndrewEichmann-NOAA merged commit 358335e into develop Mar 11, 2025
@AndrewEichmann-NOAA AndrewEichmann-NOAA deleted the feature/jcb-obs-list branch March 11, 2025 17:04
jspsl pushed a commit that referenced this pull request Mar 26, 2025
# Description

This converts the prepoceanobs task to using an obs list that ingests
the jinja yaml observation configs in jcb-gdas. Also attempts to align
script-internal variable names with existing JEDI yaml nomenclature.
Similar changes will also be incorporated into
NOAA-EMC/global-workflow#3381

# Companion PRs

None

# Issues

Resolves #1285

# Automated CI tests to run in Global Workflow

- [x] C48mx500_3DVarAOWCDA <!-- JEDI low-res marine 3DVar cycled DA !-->
- [x] C48mx500_hybAOWCDA <!-- JEDI marine hybrid envar cycled DA !-->
danholdaway added a commit that referenced this pull request Mar 26, 2025
* develop:
  added original date (#1570)
  Update OCEAN_BASIN_FILE_ORION path for marine ctest (#1569)
  Fixing window_begin and end in yaml for redating ADT (#1565)
  Save LETKF mean analysis and mean increment for verification (#1566)
  Update JEDI hashes (20250324) (#1567)
  Removes obsolete json files from marine bufr2ioda (#1564)
  Add hybrid 4denvar as a phase3 validation configuration (#1552)
  Bugfix/ocn ecen (#1550)
  Bugfix SSS obs for LETKF (#1560)
  Bugfix ocean ensemble recentering (#1558)
  apply ci path patch only when running g-w ctests (#1554)
  Update JEDI hashes (20250317)  (#1531)
  add option to create_experiment.py in create_exp.sh  (#1546)
  Update jcb hashes and support initial IODA stats capability (#1547)
  Fix GW-CI CTest dependencies (#1534)
  Converts prepoceanobs to use JCB-style obs list and config files (#1527)
  JEDI-based ensemble recentering and analysis calculation (#1488)
  tools for phase3 validation (#1526)
  Update JEDI hashes (20250306) (#1520)
  New jcb-gdas # and insitu obs prep bug fix  (#1523)
  Marine in situ obs update (SSS, moorings, drifters) (#1511)
  Save marine background and analysis spread (#1517)
  Minor build and install updates and ctest cleanup (#1512)
  correct logic for upper loop limit in chem_diagb (#1516)
  Incorporate multi-cycle concatination and separation of variables in post-BUFR observations (#1504)
  Update JEDI hashes (20250225) (#1475)
  Add a depth dependent e-fold scale (#1494)
  Update jcb and da-utils hashes to support land DA. (#1509)
  Add options to inflate the ensemble while recentering (#1501)
  Add components to use jcb yamls for marine letkf job (#1486)
  Add GNSSRO bufr2ioda and json files (#1448)
  Removal of the marine verification tools (#1499)
  update logic to identify MSU machine (#1498)
  IODA Converter with In-Situ Observations Concatenation and ObsError Inflation (#1472)
  Add sea ice recentering (#1478)
  marine vrfy bugfix (#1483)
  add LD_LIBRARY_PATH patch for wcoss2 build (#1484)
  Bugfix for ensemble recenter + add a test (#1477)
  Add code changes for the implementation of the  low-resolution B (#1441)
  update spack-stack path in Hera modulefiles (#1471)
  fixed glider output filename (#1469)
  add gdas_waveinit to C48mx500_3DVarAOWCDA ctest suite (#1470)
  update jcb-gdas hash 20250128 (#1466)
  Time series of omb stats and html doc (#1463)
  Update marine vrfy HTML codes (#1464)
  Automate the verify tools to submit from start pdy to end pdy (#1458)
  Update an offline marine vrfy tool (#1447)
  Update JEDI hashes (20250127) (#1460)
DavidNew-NOAA pushed a commit that referenced this pull request Jan 16, 2026
# Description

This converts the prepoceanobs task to using an obs list that ingests
the jinja yaml observation configs in jcb-gdas. Also attempts to align
script-internal variable names with existing JEDI yaml nomenclature.
Similar changes will also be incorporated into
NOAA-EMC/global-workflow#3381

# Companion PRs

None

# Issues

Resolves #1285

# Automated CI tests to run in Global Workflow

- [x] C48mx500_3DVarAOWCDA <!-- JEDI low-res marine 3DVar cycled DA !-->
- [x] C48mx500_hybAOWCDA <!-- JEDI marine hybrid envar cycled DA !-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

hera-GW-RT-Passed Automated testing with global-workflow successful on Hera

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Use jinja template for the obs list

4 participants